Moving Fast and Slow: Analysis of Representations and Post-Processing in Speech-Driven Automatic Gesture Generation

This paper presents a novel framework for speech-driven gesture production, applicable to virtual agents enhance human-computer interaction. Specifically, we extend recent deep-learning-based, data-driven methods generation by incorporating representation learning. Our model takes speech as input and produces gestures output, in the form of sequence 3D coordinates. We provide an analysis different representations (speech) output (motion) network both objective subjective evaluations. also analyse importance smoothing produced motion. results indicated that proposed method improved on our baseline terms measures. For example, it better captured motion dynamics matched motion-speed distribution. Moreover, performed user studies two datasets. The confirmed is perceived more natural than baseline, although difference was eliminated appropriate post-processing: hip-centering smoothing. conclude important take post-processing into account when designing automatic gesture-production method.